Analysis of Nonlinear Effects and Their Mitigation in Fiber-optic Communication Systems

The rapid development of fiber optic communication systems requires higher transmission data rate and longer reach. This thesis deals with the limiting factors in design of long-haul fiber optic communication systems and the techniques used to suppress their resulting impairments. These impairments include fiber chromatic dispersion, the Ker nonlinearity and nonlinear phase noise due to amplified spontaneous emission. In the first part of this thesis, we investigate the effect of amplified spontaneous noise in quasi-linear systems. In quasi-linear systems, inline optical amplifiers change the amplitude of the optical field envelope randomly and fiber nonlinear effects such as self phase modulation (SPM) convert the amplitude fluctuations to phase fluctuations which is known as nonlinear phase noise. For M-ary phase shift keying (PSK) signals, symbol error probability is determined solely by the probability density function (PDF) of the phase. Under the Gaussian PDF assumption, the phase variance can be related to the symbol error probability for PSK signals. We implemented the simulation based on analytical phase noise variance and Monte-Carlo simulation, and it is found that the analytical approximation is in good agreement with numerical simulations. We have developed analytical expressions for the linear and nonlinear phase noise variance due to SPM using second-order perturbation theory. It is found iv that as the transmission reach and/or lunch power increase, the variance of the phase noise calculated using first order perturbation theory becomes inaccurate. However, the variance calculated using second order perturbation theory is in good agreement with numerical simulations. We have also showed that the analytical formula given in this chapter for the variance of nonlinear phase noise can be used as a design tool to investigate the optimum system design parameters such as average power and dispersion maps for coherent fiber optic systems based on phase shift keying due to the fact that the numerical simulation of nonlinear Schrodinger (NLS) equation is time consuming, however, the analytical method based on solving NLS equation using perturbation approximation is quite efficient and therefore the analytical variance can be obtained more easily without requiring extensive computational efforts, and also with fairly good accuracy. In the second part of this thesis, an improved optical signal processing using highly nonlinear fibers is studied. This technique, optical backward propagation (OBP), can compensate for the fiber dispersion and nonlinearity using optical nonlinearity compensators (NLC) and dispersion compensating fibers (DCF), respectively. In contrast, digital backward propagation (DBP) uses the high-speed digital signal processing (DSP) unit to compensate for the fiber nonlinearity and dispersion digital domain. NLC imparts a phase shift that is equal in magnitude to the nonlinear phase shift due to Fiber propagation, but opposite in sign. In principle, BP schemes could undo the deterministic (bit-pattern dependent) nonlinear impairments, but it can not compensate for the stochastic nonlinear impairments such as nonlinear phase noise. We also introduced a novel inline optical nonlinearity compensation (IONC) technique. Our Numerical simulations show that the transmission performance can be greatly